Deep Reinforcement Learning with Importance Weighted A3C for QoE enhancement in Video Delivery Services

Adaptive bitrate (ABR) algorithms are used to adapt the video bitrate based on the network conditions to improve the overall video quality of experience (QoE). Recently, reinforcement learning (RL) and asynchronous advantage actor-critic (A3C) methods have been used to generate adaptive bit rate algorithms and they have been shown to improve the overall QoE as compared to fixed rule ABR algorithms. However, a common issue in the A3C methods is the lag between behaviour policy and target policy. As a result, the behaviour and the target policies are no longer synchronized which results in suboptimal updates. In this work, we present ALISA: An Actor-Learner Architecture with Importance Sampling for efficient learning in ABR algorithms. ALISA incorporates importance sampling weights to give more weightage to relevant experience to address the lag issues with the existing A3C methods. We present the design and implementation of ALISA, and compare its performance to state-of-the-art video rate adaptation algorithms including vanilla A3C implemented in the Pensieve framework and other fixed-rule schedulers like BB, BOLA, and RB. Our results show that ALISA improves average QoE by up to 25%-48% higher average QoE than Pensieve, and even more when compared to fixed-rule schedulers.Comment: Number of pages: 10, Number of figures: 9, Conference name: 24th IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM

A retrospective analysis of adverse drug reaction reported in a tertiary care hospital

Background: The adverse drug reactions (ADRs) reported to pharmacovigilance centre in tertiary care hospital was analysed to find out the incidence and causality.Methods: This was a retrospective study to analyse the ADR reported at pharmacovigilance centre after ethical clearance from Institutional Ethic Committee (IEC). ADR data were analysed and ADRs were categorized as department-wise, system affected and causative drug. The causality of each ADR was assessed by WHO-UMC scale.Results: The majority of patients who had suffered from ADRs were between 19-64 years of age (94.2%) and male patients (58.6%) were affected more than female (41.4%). Pulmonary medicine department has reported highest number of ADR followed by dermatology department. Skin (46.5%) was most affected system followed by gastrointestinal (30.45%), CNS (21.26%), respiratory (9.0%) and remaining systems. Rifampicin (13.79%) shows the largest numbers of ADR followed by zidovudine (13.21%), nevirapine (12.64%) and diclofenac sodium (8.0%). The maximum ADRs reported were probable (94.8%) followed by possible (5.2%).Conclusions: In conclusion, the skin was most affected system followed by gastrointestinal, central nervous and respiratory system. Rifampicin has caused maximum ADRs followed by zidovudine, nevirapine and diclofenac sodium. The causality analyses showed that majority of ADRs were probable (94.8%) while remaining falls in possible (5.2%) category

Application of digital image analysis for monitoring the behavior of factors that control the rock fragmentation in opencast bench blasting: A case study conducted over four opencast coal mines of the Talcher Coalfields, India

Drilling and blasting play a very important role in driving the economy of opencast mines, as various mining activities related to strata handling are dependent on the size of the rock mass created due to blasting. Thus the analysis of fragments created from rock explosion is essential in order to monitor its compatibility with the deployed mining machineries/HEMMs (such as shovel, dumper, dragline, etc.). As over fragmentation as well as under fragmentation both tend to increase the cost of mining, the generation of fragment size in the desired range is necessary. Several factors control the rock fragmentation in blasting, such as the burden, bench height/drilling depth, stemming column, powder factor and hole diameter. The assessment of rock fragmentation with respect to the aforementioned parameters helps to enhance the blast performance and, hence, this study intends to carry out digital image analysis for monitoring the mean fragment size and boulder percentage. A highly consistent result has been obtained using forty blasting datasets carried out in the four different opencast mines of the Talcher Coalfield (India), namely Balram OCP, Ananta OCP, Lakhanpur OCP, and Lajkura OCP. Keywords: Digital image analysis, Rock fragmentation, Bench blasting, Mean fragment size, Boulder percentag

Taxonomy of Coding Techniques for Efficient Network Communications

Internet Research Task Force, Request For Comments (RFC) 8406, https://datatracker.ietf.org/doc/rfc8406/This document summarizes recommended terminology for Network Coding concepts and constructs. It provides a comprehensive set of terms in order to avoid ambiguities in future IRTF and IETF documents on Network Coding. This document is the product of the Coding for Efficient Network Communications Research Group (NWCRG), and it is in line with the terminology used by the RFCs produced by the Reliable Multicast Transport (RMT) and FEC Framework (FECFRAME) IETF working groups

Systematic network coding for lossy line networks

Esta tesis doctoral se centra en esquemas de codificación de red sistemáticos (SNC por sus siglas en inglés para systematic network coding) a nivel de paquete para proporcionar resistencia a la pérdida de paquetes en redes lineales con pérdidas. En la teoría, la codificación de red ( network coding ) es conocida por la mejora en rendimiento y fiabilidad en redes con pérdidas. Sin embargo, la traducción de la teoría del network coding en soluciones prácticas comprende algunos desafíos críticos. Esta tesis aborda estos desafíos e investiga soluciones de network coding que puedan ser usadas en la práctica para diferentes instancias de redes lineales con pérdidas. Los objetivos principales de esta tesis doctoral son: 1) desarrollar un modelo matricial que permita el tratamiento analítico de network coding para redes con pérdidas, 2) investigar de manera semi-analítica el rendimiento alcanzable y la fiabilidad para redes lineales, un simple pero útil modelo de red conceptual, 3) desarrollar esquemas prácticos de network coding para redes lineales que superen significativamente el rendimiento del estado del arte en esquemas basados puramente en la correción de borrado hacia adelante (FEC por sus siglas en ingles para forward erasure correction), y 4) estar en línea con los esfuerzos del equipo de trabajo de la investigación en internet, Internet Research Task Force (IRTF) y presentar contribuciones. Las contribuciones de esta tesis, tal que se cumplen los objetivos son las siguientes. Primero, investigamos el uso de SNC en redes con pérdida de un solo salto. Desarrollamos un modelo matricial para este caso sin re-codificar en la red. Esto nos permite comparar códigos separables de máxima distancia (MDS por su sigla en inglés) con SNC cuando se usan únicamente como FEC. Derivamos la mínima distancia de SNC y mostramos que SNC puede proporcionar fiabilidad tan cercana al MDS como se desee y lo permita el tamaño del campo. Simulamos aplicaciones prácticas a nivel de capa de aplicación de la pila de protocolos con dos resultados concretos. Primero, se muestra que utilizando decodificación progresiva de SNC se alcanzan retardos más bajos que con un código MDS y segundo, se obtiene una distribución óptima del ancho de banda para la tasa de network coding mientras se aplica SNC in redes con bandas limitadas. Segundo, investigamos la aplicación de SNC en redes de dos saltos con pérdidas. Extendemos el modelo matricial para redes con un nodo intermedio. Usando el planteamiento semi-análitico, estudiamos y caracterizamos la fiabilidad y tasa alcanzable como una función de la tasa de network coding y de la capacidad de la red. Simulamos las aplicaciones prácticas en la capa de enlace del estándar Digital Video Broadcasting via Satellite-Second Generation (DVB-S2). Proponemos un marco con arquitectura y encapsulamiento tal que se pueda usar network coding en protocolos de la capa de enlace del DVB-S2. Tercero, extendemos el modelo matricial para una red con varios nodos intermedios. Esto nos permite entender el marco matemático de mapear entidades de comunicaciones con entidades matemáticas en diferentes nodos intermedios de la red. Analizamos la fiabilidad, las tasas alcanzables, el retardo y la complejidad de los esquemas de network coding de manera semi-analítica y probamos que nuestros resultados están en línea con los resultados de la teoría de la información. Finalmente, desarrollamos un esquema inteligente de re-codificación de network coding que incluye la planificación de paquetes en los nodos intermedios. Nuestra propuesta proporciona menor retardo y menor complejidad comparada con el estado del arte en esquemas de network coding.This dissertation focuses on packet-level systematic network coding (SNC) schemes to provide resilience to packet losses in lossy line networks. In theory, network coding is known to improve throughput and reliability of lossy networks. However, the translation of the network coding theory into efficient practical network coding solutions involves some critical challenges. This dissertation addresses those challenges and investigates on network coding solutions that can be utilized in practice for different instances of the lossy line networks. The main objectives of this dissertation are: 1) to develop a matricial model that allows analytical treatment of network coding for lossy networks, 2) semi-analytical investigation of achievable throughput and reliability for line networks, a simple yet useful conceptual network model, 3) to develop practical network coding schemes for line networks that significantly outperform state-of-the-art purely forward erasure correction (FEC)-based schemes and 4) to be in line with Internet Research Task Force (IRTF) efforts and eventually contribute. The contributions of this thesis, such that the objectives are met are as follows. First, we investigate the application of SNC in one-hop lossy networks. We develop a matricial model for the case without re-encoding in the network. This allows us to compare maximum distance separable (MDS) codes with SNC when used as FEC only. We derive the minimum distance of SNC and show that SNC can provide as closed as wished to MDS reliability as the field sizes is allowed to grow. We simulate practical applications at application layer of the protocol stack with two concrete results. First, it is shown that by using progressive decoding SNC achieves smaller delay than the MDS code and second, an optimal bandwidth distribution for network coding rate is obtained while applying SNC in band-limited networks. Second, we investigate the application of SNC in two-hop lossy networks. We extend the matricial model for the networks with one intermediate node. Using the semi-analytical approach, we study and characterize the reliability and achievable rate as a function of network coding rate and capacity of the network. We simulate practical applications at link layer of Digital Video Broadcasting via Satellite-Second Generation (DVB-S2). We propose an architectural and encapsulation framework so that network coding can be used over the state-of-the-art protocols at link layer of DVB-S2. Third, we extend the matricial model for the network with several intermediate nodes. This allows us to understand the mathematical framework of mapping communication entities to mathematical entities at different intermediate nodes of the network. We analyze semi-analytically reliability, achievable rates, delay and complexity of network coding schemes and prove that our results are inline with information theoretical results. Finally, we develop a smart re-encoding network coding scheme that includes packet scheduling at the intermediate nodes. Our proposal is shown to provide smaller delay and smaller complexity than state-of-the-art network coding schemes

PPO-ABR: Proximal Policy Optimization based Deep Reinforcement Learning for Adaptive BitRate streaming

Providing a high Quality of Experience (QoE) for video streaming in 5G and beyond 5G (B5G) networks is challenging due to the dynamic nature of the underlying network conditions. Several Adaptive Bit Rate (ABR) algorithms have been developed to improve QoE, but most of them are designed based on fixed rules and unsuitable for a wide range of network conditions. Recently, Deep Reinforcement Learning (DRL) based Asynchronous Advantage Actor-Critic (A3C) methods have recently demonstrated promise in their ability to generalise to diverse network conditions, but they still have limitations. One specific issue with A3C methods is the lag between each actor's behavior policy and central learner's target policy. Consequently, suboptimal updates emerge when the behavior and target policies become out of synchronization. In this paper, we address the problems faced by vanilla-A3C by integrating the on-policy-based multi-agent DRL method into the existing video streaming framework. Specifically, we propose a novel system for ABR generation - Proximal Policy Optimization-based DRL for Adaptive Bit Rate streaming (PPO-ABR). Our proposed method improves the overall video QoE by maximizing sample efficiency using a clipped probability ratio between the new and the old policies on multiple epochs of minibatch updates. The experiments on real network traces demonstrate that PPO-ABR outperforms state-of-the-art methods for different QoE variants

Reliable Adaptive Video Streaming Driven by Perceptual Semantics for Situational Awareness

A novel cross-layer optimized video adaptation driven by perceptual semantics is presented. The design target is streamed live video to enhance situational awareness in challenging communications conditions. Conventional solutions for recreational applications are inadequate and novel quality of experience (QoE) framework is proposed which allows fully controlled adaptation and enables perceptual semantic feedback. The framework relies on temporal/spatial abstraction for video applications serving beyond recreational purposes. An underlying cross-layer optimization technique takes into account feedback on network congestion (time) and erasures (space) to best distribute available (scarce) bandwidth. Systematic random linear network coding (SRNC) adds reliability while preserving perceptual semantics. Objective metrics of the perceptual features in QoE show homogeneous high performance when using the proposed scheme. Finally, the proposed scheme is in line with content-aware trends, by complying with information-centric-networking philosophy and architecture